Skip to content

chore(catalog): rebuild the object database with the current import - #702

Draft
mrosseel wants to merge 2 commits into
brickbots:mainfrom
mrosseel:chore/rebuild-objects-db
Draft

mrosseel wants to merge 2 commits into
brickbots:mainfrom
mrosseel:chore/rebuild-objects-db

Conversation

@mrosseel

@mrosseel mrosseel commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The committed astro_data/pifinder_objects.db was built on 2026-07-31 (#564). The import changes #671, #672, #675, #676, #677, #680, #681, #683, #685, #691, #692, #694 and #695 came after it, so devices do not get them. The Named Stars catalog of #695 is not in the committed file. This PR rebuilds the file with python -m PiFinder.catalog_imports.main on Python 3.9 with requirements.txt. Two builds with different PYTHONHASHSEED values are byte-identical.

What changed in the data

  • Catalogs: 21 -> 22 (Nam is new: 578 stars, for example Vega = Nam 91262).
  • Objects: 149329 -> 146632. Stars are merged by Hipparcos number, and Steinicke duplicates are attached to their object.
  • EGC: 91 -> 102 (the Magellanic Cloud globulars). SaM: 2162 -> 2157 (the five pairs SAC marks rejected are skipped). All other catalogs keep their counts.
  • Types: the D* rule types 40387 WDS listings as single stars (D* 134158 -> 91444, * 742 -> 41464). The Barnard objects are DN, not Nb (DN 3 -> 346). IC 1805 = Sh2 190 = Lyn 77 is one C+N object (was OC), and M 16 is C+N.
  • A double star is filtered by its brightest magnitude (about 127000 WDS listings change). Harris magnitudes replace the other values for the globulars. M 8, M 17, M 20, M 43 and M 78 now have a magnitude.
  • 257 Sharpless objects move by more than 1 arcminute, because the minutes of a south declination now have the correct sign (Sh2 5 moves by about 4 degrees).
  • The cited corrections in astro_data/corrections/ apply.

Magnitude decimals

The Harris and Lynga loaders parsed their float columns as float32, and arp.sqlite stores its magnitudes as float32 values. The database held values such as 6.46999979019165 for M 2 (Harris V = 6.47), in 147 Harris, 139 Lynga and 71 Arp objects. The Harris and Lynga loaders now parse float64, and the Arp loader rounds to two decimals. No magnitude in the rebuilt database has float noise. A test checks M 2 = 6.47.

Tests

  • tests/test_catalog_data.py reads the committed file. It is updated with it: the number of catalogs, the Nam, EGC and SaM counts, and the type of IC 1805.
  • The catalog and database tests pass (605). The smoke, unit and UI module tests pass, as do mypy and ruff 0.4.8 check and format.

🤖 Generated with Claude Code

mrosseel and others added 2 commits November 18, 2026 03:30
The committed astro_data/pifinder_objects.db was built on 2026-07-31
(brickbots#564). The import changes brickbots#671, brickbots#672, brickbots#675, brickbots#676, brickbots#677, brickbots#680, brickbots#681,
brickbots#683, brickbots#685, brickbots#691, brickbots#692, brickbots#694 and brickbots#695 came after it, so the device data
does not show them. The Named Stars catalog of brickbots#695 is not in the
committed file at all. This commit rebuilds the file with
python -m PiFinder.catalog_imports.main on Python 3.9 with
requirements.txt. Two builds with different PYTHONHASHSEED values are
byte-identical.

Counts, before -> after:
- catalogs: 21 -> 22 (Nam is new)
- objects: 149329 -> 146632 (stars merged by Hipparcos number,
  Steinicke duplicates attached to their object)
- catalog listings: 151170 -> 151754
- Nam: 0 -> 578 (Vega is Nam 91262)
- EGC: 91 -> 102 (the Magellanic Cloud globulars)
- SaM: 2162 -> 2157 (the five pairs SAC marks rejected are skipped)
- all other catalogs keep their counts

Object types, before -> after:
- D*: 134158 -> 91444 and *: 742 -> 41464 (the D* rule types 40387
  WDS listings as single stars)
- DN: 3 -> 346 (the Barnard objects are dark nebulae, not Nb)
- Nb: 842 -> 506, Gx: 10747 -> 10123, OC: 1621 -> 1524,
  C+N: 82 -> 89, Gb: 251 -> 261
- IC 1805, Sh2 190 and Lyn 77 are one object of type C+N (was OC).
  M 16 is C+N (was OC).

Other notable changes:
- A double star is filtered by its brightest magnitude. This changes
  the filter magnitude of about 127000 WDS listings and most SaM pairs.
- The Harris magnitudes replace the other values for the globulars
  (M 2: 6.6 -> 6.47, M 69: 8.3 -> 7.64).
- M 8, M 17, M 20, M 43 and M 78 have a magnitude (none before).
- 257 Sharpless objects move by more than 1 arcminute: the minutes of
  a south declination now have the correct sign (Sh2 5 moves by 4
  degrees).
- The cited corrections in astro_data/corrections/ apply.

The data assertions in tests/test_catalog_data.py read the committed
file. They change with it: the number of catalogs, the Nam, EGC and
SaM counts, and the type of IC 1805.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Harris and Lynga loaders parsed their float columns as float32, and
arp.sqlite stores its magnitudes as float32 values. So the database held
values such as 6.46999979019165 for M 2 (Harris V = 6.47), in 147 Harris,
139 Lynga and 71 Arp objects. The Harris and Lynga loaders now parse
float64, and the Arp loader rounds to two decimals. The database is
rebuilt; no magnitude has float noise now. Two builds with different
PYTHONHASHSEED values are byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant